cagent with Tiny Language Models and Skills: Introduction

I was recently telling you about cagent, in the article Using cagent with gpt-oss-120b and OVH AI endpoints where I explained how to use cagent with large language models (LLMs) like gpt-oss-120b hosted by OVHCloud.

But it’s also possible to use cagent with local LLMs using Docker Model Runner, and given my machine, I stick to models under 8b parameters. This way I ensure good responsiveness and a smooth user experience.

So yes, models under 8b parameters (the “tiny language models”) don’t have the same capabilities as Claude.ai, but we can still find them useful in our daily activities. You need to know their constraints and limitations, and use them wisely.

I’m going to write several articles to explain how to use cagent with small local LLMs, and how to get the most out of these models for different tasks. Today, let’s forget about code generation — we’re going to talk about cooking.

My goal is to have, no longer a code agent, but a cooking agent that will help me with my meal preparation projects.

Which LLM to choose?

That’s always the tricky question — there’s no magic answer (and no magic recipe either). You need to stay on the lookout, test different models, and find the one that best fits your needs and your hardware.

Today, I’m going to use this one: Jan-nano-128k-gguf, for several reasons:

• It’s not too big: 4b parameters.
• It has a context window of 128k tokens, which will help with its ability to swallow prompts.
• It has excellent tool support (for function calling), but we’ll only use that in a future article.

Is Jan-Nano a good cook? Not really, but we’re going to see how to help it become one, and how to make it do interesting things in the kitchen.

Setting up the work environment

I usually set up my projects with devcontainer (and with agents, I think it’s better) coupled with Docker Compose. You’ll find the configuration here: https://codeberg.org/cagent-cookbook/01-initialize/src/branch/main/.devcontainer. The Dockerfile contains everything needed, including the installation of cagent.

I also use Docker Compose to automatically download the LLM.

compose.yml:

services:

  cagent-01-initialize-workspace:
    build:
      context: .
      dockerfile: Dockerfile
    network_mode: "host"
    volumes:
      - ..:/workspaces:cached
    command: sleep infinity

    models:
      jan-nano:

models:

  jan-nano:
    model: huggingface.co/menlo/jan-nano-128k-gguf:Q4_K_M
    context_size: 16384

Otherwise you can do it manually with the following command:

docker model pull huggingface.co/menlo/jan-nano-128k-gguf:Q4_K_M

Configuring cagent to “do the cooking”

cagent is configured using a yaml configuration file, here’s the one we’re going to use:

config.yaml:

agents:
  root:

    model: cooking
    num_history_items: 5
    description: Bob, Expert Home Cook Mentor
    instruction: |
      You name is Bob.
      # Role: Expert Home Cook Mentor
      You are a warm, knowledgeable, and highly practical Home Cook Mentor. Your goal is to make cooking accessible, safe, and successful for everyone, regardless of skill level.

      ## Guidelines & Behavior:
      1. **Simplicity First:** Use plain, clear English. If you use a technical term (e.g., "deglazing"), provide a brief, one-sentence explanation.
      2. **Organization:** Always structure recipes clearly:
          - **Pantry Staples:** (Oil, salt, pepper, common spices)
          - **Fresh Items:** (Produce, proteins, dairy)
          - **Numbered Steps:** Concise, chronological instructions.
      3. **Sensory Cues:** Don't just give times; describe smells, textures, and sights (e.g., "Sauté until the onions are translucent and smell sweet, about 5 minutes").
      4. **Elevate the Dish:** End every recipe with a "Mentor's Tip"—a simple way to improve flavor (e.g., a splash of vinegar, a pinch of flaky salt, or a fresh herb garnish).

      ## Safety & Accuracy (Priority):
      - Always mention internal temperature targets for proteins (e.g., 165°F/74°C for chicken).
      - Remind users of safety basics when relevant (e.g., "Wash your hands after handling raw meat").
      - If a request is dangerous or nonsensical, gently correct the user and suggest a safe alternative.

      ## Formatting Rules:
      - Use **bolding** for ingredients and key temperatures.
      - Use `###` for section headers.
      - Keep responses concise to stay within your context window. Avoid long preambles; get straight to the cooking.

models:

  cooking:
    provider: dmr
    model: huggingface.co/menlo/jan-nano-128k-gguf:Q4_K_M

Note: num_history_items: 5 — I limited the history to 5 items to avoid saturating the model’s context window, and this will help the agent stay focused. Let’s not forget that we’re working with a small model, so we need to be careful not to give it too much information to process at once.

Running the agent

Now that everything is ready, all that’s left is to launch the agent in a VSCode terminal:

cagent run config.yaml

I’m assuming you’ve launched your project with devcontainer.

You should see the cagent interface open in your terminal,

and you can start chatting with Bob, your cooking mentor.

As the conversations go on, you’ll probably realize that Bob is no Michelin-starred chef. So we’re going to have to help him out a bit, by providing him with recipes for instance. And for that, we’re going to use the concept of skills.

cagent and skills

First, we’re going to enable skills in the agent configuration by adding the following line: skills: true:

agents:
  root:

    model: cooking
    skills: true
    num_history_items: 5
    ...

I also added this to the agent’s instructions:

## Skills:
- Always check for existing related skills before ansering a question.
- If a relevant skill exists, use it instead of answering directly with your base knoledge.

This way, Bob should understand that he needs to first check if he has a skill matching the question, before answering with his base knowledge.

Creating skills

So at the root of the project, we’re going to create an .agents folder and a skills subfolder:

├── .agents
│   └── skills

And the “skills” will of course go in this skills folder.

Skill format — 1st recipe

A skill is a markdown file named SKILL.md that contains the description of the competency we want to teach the agent. It lives in a folder named after the competency. For example, for the first recipe, we’ll create a kung-pao-chicken skill:

├── .agents
│   └── skills
│       └── kung-pao-chicken
│           └── SKILL.md

The SKILL.md file starts with a metadata header section, with the skill name (matching the directory name) and a description of the competency, followed by a detailed description of the recipe (in our case).

kung-pao-chicken/SKILL.md:

---
name: kung-pao-chicken
description: Kung Pao Chicken is a classic Sichuan dish featuring tender chicken pieces.
---

# Kung Pao Chicken

## Summary
Kung Pao Chicken is a classic Sichuan dish featuring tender chicken pieces stir-fried with peanuts, vegetables, and dried chilies in a sweet, sour, and spicy sauce. This bold and flavorful dish delivers the perfect balance of heat and savory notes.

...

You’ll find the full recipe in this file https://codeberg.org/cagent-cookbook/01-initialize/raw/branch/main/.agents/skills/kung-pao-chicken/SKILL.md

I created a second skill for the Peking Duck recipe:

peking-duck/SKILL.md:

---
name: peking-duck
description: Peking Duck is an iconic Chinese dish from Beijing, famous for its crispy skin and tender meat.
---

# Peking Duck

## Summary
Peking Duck is an iconic Chinese dish from Beijing, famous for its crispy skin and tender meat. Traditionally served with thin pancakes, scallions, cucumber, and hoisin sauce, this impressive dish requires time and technique but delivers spectacular results.

...

You’ll find the full recipe in this file https://codeberg.org/cagent-cookbook/01-initialize/raw/branch/main/.agents/skills/peking-duck/peking-duck.md

Using skills

You need to restart cagent for the skills to be picked up, and then you can ask Bob to give you the Kung Pao Chicken or Peking Duck recipe, and he should be able to provide it.

You can notice on the right that the agent has detected 2 skills:

And when I ask for an explanation of how to make Peking Duck, Bob reads the corresponding skill:

Then generates the Peking Duck recipe from the skill:

And you can then ask him about the recipe, for example the list of ingredients:

And of course, you can do the same with the Kung Pao Chicken recipe:

You can also activate a skill directly using the / <skill_name> command:

See, it’s simple. In a next article, we’ll see how to use tools for function calling and for example save recipes to a file. We’ll also see how to trigger processing with skills.

To be continued…

Source code for this article: https://codeberg.org/cagent-cookbook/01-initialize